How to train a self-driving vehicle: On the added value (or lack thereof) of curriculum learning and replay buffers.

Learning from only real-world collected data can be unrealistic and time consuming in many scenario. One alternative is to use synthetic data as learning environments to learn rare situations and replay buffers to speed up the learning. In this work, we examine the hypothesis of how the creation of the environment affects the training of reinforcement learning agent through auto-generated environment mechanisms. We take the autonomous vehicle as an application. We compare the effect of two approaches to generate training data for artificial cognitive agents. We consider the added value of curriculum learning-just as in human learning-as a way to structure novel training data that the agent has not seen before as well as that of using a replay buffer to train further on data the agent has seen before. In other words, the focus of this paper is on characteristics of the training data rather than on learning algorithms. We therefore use two tasks that are commonly trained early on in autonomous vehicle research: lane keeping and pedestrian avoidance. Our main results show that curriculum learning indeed offers an additional benefit over a vanilla reinforcement learning approach (using Deep-Q Learning), but the replay buffer actually has a detrimental effect in most (but not all) combinations of data generation approaches we considered here. The benefit of curriculum learning does depend on the existence of a well-defined difficulty metric with which various training scenarios can be ordered. In the lane-keeping task, we can define it as a function of the curvature of the road, in which the steeper and more occurring curves on the road, the more difficult it gets. Defining such a difficulty metric in other scenarios is not always trivial. In general, the results of this paper emphasize both the importance of considering data characterization, such as curriculum learning, and the importance of defining an appropriate metric for the task.

Inflammatory and metabolic markers in relation to outcome of in vitro fertilization in a cohort of predominantly overweight and obese women.

For overweight and obese women undergoing in vitro fertilization (IVF) the pregnancy and live birth rates are compromised while the underlying mechanisms and predictors are unclear. The aim was to explore the association between adipose tissue-related inflammatory and metabolic markers and the pregnancy and live birth outcome of IVF in a cohort of predominantly overweight and obese women. Serum samples, fulfilling standardizing criteria, were identified from 195 women having participated in either the control (n = 131) or intervention (n = 64) group of a randomized controlled trial (RCT), seeking to evaluate the effect of a weight reduction intervention on IVF outcome in obese women. Serum high-sensitivity C-reactive protein (hsCRP) and the adipokines leptin and adipocyte fatty acid-binding protein (AFABP) were analyzed for the whole cohort (n = 195) in samples collected shortly before IVF [at randomization (control group), after intervention (intervention group)]. Information on age, anthropometry [BMI, waist circumference, waist-to-height ratio (WHtR)], pregnancy and live birth rates after IVF, as well as the spontaneous pregnancy rate, was extracted or calculated from collected data. The women of the original intervention group were also characterized at randomization regarding all variables. Eight women [n = 3 original control group (2.3%), n = 5 original intervention group (7.8%)] conceived spontaneously before starting IVF. BMI category proportions in the cohort undergoing IVF (n = 187) were 1.6/20.1/78.3% (normal weight/overweight/obese). The pregnancy and live birth rates after IVF for the cohort were 35.8% (n = 67) and 24.6% (n = 46), respectively. Multivariable logistic regression revealed that none of the variables (age, hsCRP, leptin, AFABP, BMI, waist circumference, WHtR) were predictive factors of pregnancy or live birth after IVF. Women of the original intervention group displayed reductions in hsCRP, leptin, and anthropometric variables after intervention while AFABP was unchanged. In this cohort of predominantly overweight and obese women undergoing IVF, neither low-grade inflammation, in terms of hsCRP, other circulating inflammatory and metabolic markers released from adipose tissue (leptin, AFABP), nor anthropometric measures of adiposity or adipose tissue distribution (BMI, waist, WHtR) were identified as predictive factors of pregnancy or live birth rate.Trial registration: ClinicalTrials.gov number, NCT01566929. Trial registration date 30-03-2012, retrospectively registered.

Longitudinal changes in adipokines and free leptin index during and after pregnancy in women with obesity.

OBJECTIVE: Detailed data on adipokines and body composition during and after pregnancy in women of different BMI categories are lacking. Furthermore, adipokine regulation during pregnancy and the factors contributing to gestational insulin resistance are not completely understood. The objective was to longitudinally determine adipokine levels, body composition, and insulin sensitivity during and after pregnancy in women of healthy weight (HW) and with obesity (OB), and identify factors associated with insulin resistance. DESIGN: Women (30 HW, 19 OB) underwent blood sampling and body composition examination, by air-displacement plethysmography, longitudinally during pregnancy (trimesters 1, 2, 3) and after pregnancy (6, 12, 18 months postpartum). Serum leptin, soluble leptin receptor (sOB-R), and adiponectin levels were measured and free leptin index (FLI) and homeostatic model assessment of insulin resistance (HOMA-IR) determined. RESULTS: Fat mass and leptin increased during pregnancy in the HW (p < 0.01) but not in the OB group. sOB-R increased during pregnancy in both groups (p < 0.001). Thus, FLI was unchanged in HW throughout pregnancy but reduced in OB (p = 0.001), although consistently higher in OB. Adiponectin decreased in both groups during pregnancy (p < 0.001 for HW, p = 0.01 for OB). After pregnancy, adiponectin increased in both groups, but more markedly in OB where it reached trimester 1 levels. Multivariable regression identified FLI as the variable most strongly associated with HOMA-IR in all trimesters, but not after pregnancy. CONCLUSIONS: Leptin, sOB-R, adiponectin, and FLI undergo marked changes during and after pregnancy with differences in women of different BMI. We suggest that leptin activity is regulated by its soluble receptor and that this is an important factor for optimizing fat mass and insulin sensitivity during pregnancy.

Integration of biological factors in the treatment plan evaluation in breast cancer radiotherapy.

BACKGROUND/PURPOSE: Tumor biology and patient smoking status have clear effects on the benefit of breast radiotherapy. This study developed treatment evaluation strategies that integrated dosimetry, tumor aggressiveness and smoking status for patients undergoing hypo-fractionated whole breast irradiation with simultaneous integrated boost. MATERIALS/METHODS: The evaluation method Plan Quality Metrics (PQM) was adapted for breast cancer. Radiotherapy (RT) benefit was assessed for three levels of tumor aggressiveness; RT risk was estimated using mean dose to organs at risk and published Excess Relative Risk per Gy data for lung cancer and cardiac mortality for smokers and non-smokers. Risk for contralateral breast cancer was also evaluated. PQM and benefit/risk was applied to four patient groups (n = 10 each). Plans using 3D conformal radiotherapy (3DCRT), 3DCRT plus intensity-modulated radiation therapy (IMRT), 3DCRT plus volumetric modulated arc therapy (VMAT) and VMAT were evaluated for each patient. RESULTS: 3DCRT-IMRT hybrid planning resulted in higher PQM score (median 87.0 vs. 3DCRT 82.4, p < 0.01), better dose conformity, lower doses to the heart, lungs and contralateral breast. Survival benefit was most predominant for patients with high-risk breast cancer (>7% and >4.5% gain for non-smokers and smokers). For smokers with intermediate- or low-risk breast cancer, RT induced mortality risk dominated for all techniques. When considering the risk of local recurrence, RT benefitted also smokers (>5% and >2% for intermediate- and low-risk cancer). CONCLUSIONS: PQM methodology was suggested for breast cancer radiotherapy evaluation. Further validation is needed. RT was beneficial for all patients with high risk of recurrence. A survival benefit for smokers with low or intermediate risk of recurrence could not be confirmed.

Adipose tissue and body composition in women six years after gestational diabetes: factors associated with development of type 2 diabetes.

Factors differentiating women at highest risk of progression to type 2 diabetes mellitus (T2DM) after gestational diabetes mellitus (GDM) are incompletely known. Our aim was to characterize adipose tissue and body composition in relation to glucose metabolism in women with a history of GDM and to identify factors associated with development of T2DM. We examined glucose tolerance (OGTT), insulin sensitivity (HOMA-IR), body composition (anthropometry, air displacement plethysmography), and blood chemistry in 39 women 6 years after GDM. An adipose tissue biopsy was obtained to assess the size, number, and lipolytic activity of adipocytes, and adipokine release and density of immune cells and blood vessels in adipose tissue. Normal glucose tolerance (NGT) was identified in 31 women and impaired glucose metabolism (IGM) in 8. Women with IGM had higher BMI/fat mass, and related expected adipose tissue features, than women with NGT. Ethnicity was similar in the groups, but numerically there was a higher proportion of European women in the NGT group and a higher proportion of non-European women in the IGM group. BMI was the best discriminator of NGT versus IGM (multivariable logistic regression: OR = 1.34, P < 0.01). Waist-to-height ratio and adipocyte volume were most strongly associated with HOMA-IR (multivariable linear regression: R2 = 0.656, P < 0.001). After adjustment for BMI/ethnicity, women with IGM had increased serum adipocyte fatty acid-binding protein, weight gain after index pregnancy, and a lower proportion of fat-free mass. These factors, together with high BMI, abdominal fat distribution, and enlarged adipocytes, may increase the risk of progression to T2DM after GDM.

Free lipid and computerized determination of adipocyte size.

The size distribution of adipocytes in a suspension, after collagenase digestion of adipose tissue, can be determined by computerized image analysis. Free lipid, forming droplets, in such suspensions implicates a bias since droplets present in the images may be identified as adipocytes. This problem is not always adjusted for and some reports state that distinguishing droplets and cells is a considerable problem. In addition, if the droplets originate mainly from rupture of large adipocytes, as often described, this will also bias size analysis. We here confirm that our ordinary manual means of distinguishing droplets and adipocytes in the images ensure correct and rapid identification before exclusion of the droplets. Further, in our suspensions, prepared with focus on gentle handling of tissue and cells, we find no association between the amount of free lipid and mean adipocyte size or proportion of large adipocytes.

A new approach to measuring vitamin D in human adipose tissue using time-of-flight secondary ion mass spectrometry: a pilot study.

Circulating concentrations of vitamin D, 25(OH)D, and 1,25(OH)2D are lower in obese than lean individuals, but little is known about the adipose tissue content of these molecules. The aim of this study was to explore the possibility to use time-of-flight secondary ion mass spectrometry (TOF-SIMS) to measure vitamin D and its metabolites in fat tissue in obese and lean subjects. Abdominal subcutaneous adipose tissue (SAT) biopsies were obtained from three lean and three obese women, and paired biopsies SAT and visceral adipose tissue (VAT) were obtained from three obese subjects during gastric bypass surgery. TOF-SIMS was used to measure vitamin D3, 25(OH)D3, and 1,25(OH)2D3 in adipose tissue. We found that vitamin D3, 25(OH)D3, and 1,25(OH)2D3 in adipose tissue can be measured with TOF-SIMS. In adipose tissue, vitamin D3 and its metabolites were located in adipocyte lipid droplets. The content of vitamin D3 (P=0.006) and 25(OH)D3 (P=0.018) were lower in SAT in obese compared with lean women. TOF-SIMS has the potential to semi-quantitatively measure vitamin D metabolites in adipose tissue, and offers a possibility to compare vitamin D levels in different depots and groups of individuals. It also gives the opportunity to explore the localization of vitamin D metabolites at a cellular level.

Adiponectin, chemerin, cytokines, and dipeptidyl peptidase 4 are released from human adipose tissue in a depot-dependent manner: an in vitro system including human serum albumin.

BACKGROUND: Adipose tissue (AT) contributes to metabolic dysfunction through imbalanced production of adipokines, including cytokines. Visceral AT in particular is associated with metabolic disorders, indicating a specific secretory status. The relative significance of different human AT depots in adipokine release is not fully known. Further, previous in vitro systems usually included medium containing bovine serum albumin (BSA), which may induce cytokine release. Our aim was to compare release of a number of adipokines/cytokines - all implicated in insulin resistance - from human subcutaneous and visceral AT in a short-term incubation system minimizing cytokine induction and including repeated measurements during 24 h. A prerequisite was to evaluate a potential alternative to BSA in the incubation medium. METHODS: Subcutaneous and/or visceral AT from 17 patients (age 20-68 years; BMI 22.6-56.7 kg/m2) undergoing elective surgery was incubated for 2, 4, 6, 8, and 24 h in medium with or without 1% BSA or human serum albumin (HSA). Medium concentrations of adiponectin, chemerin, nine cytokines, dipeptidyl peptidase 4 (DPP4), and omentin were analyzed by multiplex immunoassay or ELISA. Adipocyte size, AT macrophage density, and medium concentrations of endotoxin were determined. RESULTS: Cytokine release was induced by BSA but not by HSA. In evaluation of the final incubation protocol including 1% HSA, and as expected, adiponectin release was higher from subcutaneous biopsies of nonobese than of obese subjects and inversely associated with adipocyte size; omentin was released almost exclusively from visceral AT. Exploratory incubations revealed more abundant release of chemerin, cytokines (except IL-6), and DPP4 from the visceral depot, while adiponectin release was higher from subcutaneous than visceral AT. Release was linear for a maximum of 2-6 h. Macrophage density was higher in visceral than subcutaneous AT. Levels of endotoxin in the medium were negligible. CONCLUSIONS: Adiponectin, chemerin, many cytokines, and DPP4 are released from human AT in a depot-dependent manner. These results highlight functional differences between visceral and subcutaneous AT, and a mechanistic link between regional fat accumulation and metabolic disorders. Supplementation of human AT incubation medium with HSA rather than BSA is recommended to minimize induction of cytokine release.

Continuous administration of a P450 aromatase inhibitor induces polycystic ovary syndrome with a metabolic and endocrine phenotype in female rats at adult age.

Studying the mechanisms for the complex pathogenesis of polycystic ovary syndrome (PCOS) requires animal models with endocrine, reproductive, and metabolic features of the syndrome. Hyperandrogenism seems to be a central factor in PCOS, leading to anovulation and insulin resistance. In female rats, continuous administration of letrozole, a nonsteroidal inhibitor of P450 aromatase, at 400 µg/d starting before puberty induces hyperandrogenemia and reproductive abnormalities similar to those in women with PCOS. However, despite high circulating testosterone levels, these rats do not develop metabolic abnormalities, perhaps because of their supraphysiological testosterone concentrations or because estrogen synthesis is completely blocked in insulin-sensitive tissues. To test the hypothesis that continuous administration of lower doses of letrozole starting before puberty would result in both metabolic and reproductive phenotypes of PCOS, we performed a 12-wk dose-response study. At 21 d of age, 46 female Wistar rats were divided into two letrozole groups (100 or 200 µg/d) and a control group (placebo). Both letrozole doses resulted in increased body weight, inguinal fat accumulation, anovulation, larger ovaries with follicular atresia and multiples cysts, endogenous hyperandrogemia, and lower estrogen levels. Moreover, rats that received 200 µg/d had insulin resistance and enlarged adipocytes in inguinal and mesenteric fat depots, increased circulating levels of LH, decreased levels of FSH, and increased ovarian expression of Cyp17a1 mRNA. Thus, continuous administration of letrozole, 200 µg/d, to female rats for 90 d starting before puberty results in a PCOS model with reproductive and metabolic features of the syndrome.

Modeling the Development of Goal-Specificity in Mirror Neurons.

Neurophysiological studies have shown that parietal mirror neurons encode not only actions but also the goal of these actions. Although some mirror neurons will fire whenever a certain action is perceived (goal-independently), most will only fire if the motion is perceived as part of an action with a specific goal. This result is important for the action-understanding hypothesis as it provides a potential neurological basis for such a cognitive ability. It is also relevant for the design of artificial cognitive systems, in particular robotic systems that rely on computational models of the mirror system in their interaction with other agents. Yet, to date, no computational model has explicitly addressed the mechanisms that give rise to both goal-specific and goal-independent parietal mirror neurons. In the present paper, we present a computational model based on a self-organizing map, which receives artificial inputs representing information about both the observed or executed actions and the context in which they were executed. We show that the map develops a biologically plausible organization in which goal-specific mirror neurons emerge. We further show that the fundamental cause for both the appearance and the number of goal-specific neurons can be found in geometric relationships between the different inputs to the map. The results are important to the action-understanding hypothesis as they provide a mechanism for the emergence of goal-specific parietal mirror neurons and lead to a number of predictions: (1) Learning of new goals may mostly reassign existing goal-specific neurons rather than recruit new ones; (2) input differences between executed and observed actions can explain observed corresponding differences in the number of goal-specific neurons; and (3) the percentage of goal-specific neurons may differ between motion primitives.

A role for ARF6 in dendritic cell podosome formation and migration.

ADP-ribosylation factor 6 (ARF6) is a widely expressed GTPase that influences both membrane traffic and actin cytoskeleton function. Its role in dendritic cells (DC) has not previously been investigated. We analysed the effect of retroviral expression of ARF6 GDP/GTP binding and other functional mutants in primary murine DC. Maturation in response to lipopolysaccharide (LPS) proceeded normally in DC expressing ARF6 mutants and production of inflammatory cytokines was similarly unaffected. Although LPS-stimulated macropinocytosis was suppressed by expression of the GTP-binding Q67L ARF6 mutant we detected no overall activation of ARF6 by LPS. The ability of immature DC to migrate towards CCL3 and to a lesser extent, of mature DC to migrate towards CCL19, was compromised by expression of either the Q67L or the GDP-binding T44N mutant. Examination of the actin cytoskeleton in these cells revealed that both mutants strongly inhibited the formation of F-actin-rich podosomes, providing a possible explanation for the effects of ARF6 mutants on DC migration. Thus, these studies identify responses in DC that require normal ARF6 function, though not necessarily further ARF6 activation. They reveal for the first time a role for ARF6 in podosome formation and demonstrate functional effects of the T44N ARF6 mutant.

Enhanced dendritic cell antigen capture via toll-like receptor-induced actin remodeling.

Microbial products are sensed through Toll-like receptors (TLRs) and trigger a program of dendritic cell (DC) maturation that enables DCs to activate T cells. Although an accepted hallmark of this response is eventual down-regulation of DC endocytic capacity, we show that TLR ligands first acutely stimulate antigen macropinocytosis, leading to enhanced presentation on class I and class II major histocompatibility complex molecules. Simultaneously, actin-rich podosomes disappear, which suggests a coordinated redeployment of actin to fuel endocytosis. These reciprocal changes are transient and require p38 and extracellular signal-regulated kinase activation. Thus, the DC actin cytoskeleton can be rapidly mobilized in response to innate immune stimuli to enhance antigen capture and presentation.

Lived experiences of being a father of an adult child with schizophrenia.

The aim of this study is to analyze and describe lived experiences of being a father of an adult child with schizophrenia. Interpretations of interviews with seven Swedish fathers of sons or daughters with schizophrenia revealed a pattern of gradually changing existential consequences. After an initial period of shock when receiving the diagnosis, a long struggle to regain control follows. The findings are presented in a structure based on eight different aspects of this struggle, which seems to be characterized by a balance between grieving and adaptation. An important conclusion is that the fathers' life-world must be attended to in professional family interventions.

Contributions of amino acid side chains to the kinetics and thermodynamics of the bivalent binding of protein L to Ig kappa light chain.

Protein L is a bacterial surface protein with 4-5 immunoglobulin (Ig)-binding domains (B1-B5), each of which appears to have two binding sites for Ig, corresponding to the two edges of its beta-sheet. To verify these sites biochemically and to probe their relative contributions to the protein L-Ig kappa light chain (kappa) interaction, we compared the binding of PLW (the Y47W mutant of the B1 domain) to that of mutants designed to disrupt binding to sites 1 and 2, using gel filtration, BIAcore surface plasmon resonance, fluorescence titration, and solid-phase radioimmunoassays. Gel filtration experiments show that PLW binds kappa both in 1:1 complexes and multivalently, consistent with two binding sites. Covalent dimers of the A20C and V51C mutants of PLW were prepared to eliminate site 1 and site 2 binding, respectively; both the A20C and V51C dimers bind kappa in 1:1 complexes and multivalently, indicating that neither site 1 nor site 2 is solely responsible for kappa binding. The A20R mutant was designed computationally to eliminate site 1 binding while preserving site 2 binding; consistent with this design, the A20R mutant binds kappa in 1:1 complexes but not multivalently. To probe the contributions of amino acid side chains to binding, we prepared 75 point mutants spanning nearly every residue of PLW; BIAcore studies of these mutants revealed two binding-energy "hot spots" consistent with sites 1 and 2. These data indicate that PLW binds kappa at both sites with similar affinities (high nanomolar), with the strongest contributions to the binding energy from Tyr34 (site 2) and Tyr36 (site 1). Compared to other protein-protein complexes, the binding is insensitive to amino acid substitutions at these sites, consistent with the large number of main chain interactions relative to side chain interactions. The strong binding of protein L to Ig kappa light chains of various species may result from the ambidextrous binding of the B1-B5 domains and the unimportance of specific side chain interactions.